Implement factorised pointwise probabilities#1331
Conversation
Benchmark Report
Computer InformationBenchmark Results |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1331 +/- ##
=======================================
Coverage 82.35% 82.35%
=======================================
Files 49 49
Lines 3502 3508 +6
=======================================
+ Hits 2884 2889 +5
- Misses 618 619 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Yeah, I think we can do that. Even in testing the functionality, I needed to implement much of the machinery needed to compute arbitrary conditional/marginal distributions. There have been longstanding issues open for each of these features in Distributions, and they probably ultimately belong there, but since that's not likely to happen soon, I think it could make sense to have a PartitionedDistributions.jl package that implements at least |
|
That sounds great! I'm very happy to help out in whatever way I can :) |
|
Once sethaxen/PartitionedDistributions.jl#5 is merged and there are some basic docs, I'll make a release and register it. The easier |
|
Okay I registered PartitionedDistributions.jl, and it'll be available once JuliaRegistries/General#153153 is merged. |
penelopeysm
force-pushed
the
py/pointwise
branch
from
aa2e6e4 to
26032e7
Compare
|
DynamicPPL.jl documentation for PR #1331 is available at: |
| distributions that can be partitioned into blocks, using PartitionedDistributions.jl. For | ||
| example, if `factorize=true`, then `y ~ MvNormal(...)` will return a vector of | ||
| log-densities, one for each element of `y`. If `factorize=false`, then the log-density for | ||
| `y ~ MvNormal(...)` will be a single scalar. |
There was a problem hiding this comment.
Might be worth noting somewhere that these factorized log-densities will only add up to the total log-density when the original density can be completely factorized into independent univariate distributions.
There was a problem hiding this comment.
Yes, that sounds good
|
I realise there aren't actually any real docs on pointwise likelihoods apart from the docstrings, but also this is quite meaningful for users, so I'll add it to the TuringLang/docs repository rather than DynamicPPL where it'd be a bit hidden. |
| If `factorize=true`, additionally attempt to provide factorised log-densities for | ||
| distributions that can be partitioned into blocks, using PartitionedDistributions.jl. | ||
|
|
||
| For example, if `factorize=true`, then `y ~ MvNormal(...)` will return a vector of | ||
| log-densities, one for each element of `y`. The `i`-th element of this vector will be the | ||
| conditional log-probability of `y[i]` given all the other elements of `y` (often denoted | ||
| `log p(y_{i} | y_{-i})`): in particular this is exactly the log-density required for | ||
| leave-one-out cross-validation. | ||
|
|
||
| In contrast, if `factorize=false`, then the log-density for `y ~ MvNormal(...)` will be a | ||
| single scalar corresponding to `logpdf(MvNormal(...), y)`. | ||
|
|
||
| Note that the sum of the factorised log-densities may not, in general, be equal to the | ||
| log-density of the full distribution: they will only be equal if the original distribution | ||
| can be completely factorised into independent components. For example, if `y ~ MvNormal(μ, | ||
| Σ)` where `Σ` is diagonal, then each element of `y` is independent and the sum of the | ||
| factorised log-densities will be equal to the log-density of the full distribution. In | ||
| contrast, if `Σ` has off-diagonal entries, then the elements of `y` are not independent. | ||
| """ |
There was a problem hiding this comment.
@sethaxen Would you be willing to sense-check my docstring?
There was a problem hiding this comment.
(If you want to review generally please feel free to as well, but it's also not your job so please don't feel obliged 🙂)
2 participants
Closes #1038